The following table summarizes functionalities of replay buffers.
| cpprb | OpenAI/Baselines | Ray/RLlib | Chainer/ChainerRL | DeepMind/Reverb | |
|---|---|---|---|---|---|
| Flexible Environment | Yes | No | No | Yes | Yes |
| Nstep | Yes | No | Yes | Yes | No |
| Parellel Exploration | Yes (Support Ape-X on single machine) | Yes (Avarages gradients of MPI processes) | Yes (Concatenates sample batches from distributed buffers) | No | Yes |
| Save/Load | Only transitions (document) | No (Maybe can pickle) |
No (Maybe can pickle. Trained policies can save/load.) |
Yes | Yes |
| Deep Learning Framework | Anything | TensorFlow 1.14 (only this version) | Anything (Helper functions for TensorFlow and PyTorch) | Chainer (maintenance only) | TensorFlow 2.3 |